[Kernel] Honor leading strides in SM90 tensorwise CUTLASS scaled_mm - #55537
[Kernel] Honor leading strides in SM90 tensorwise CUTLASS scaled_mm#55537jackLei0901 wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (4)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📝 SummarySummary by CodeRabbit
WalkthroughThe CUTLASS 3.x scaled matrix multiplication callers now preserve runtime leading strides for supported padded views. Entry points validate A-stride alignment. Tests cover int8, FP8, strided views, and invalid alignment. ChangesCUTLASS stride propagation
Estimated code review effort: 2 (Simple) | ~15 minutes Merge Risk: ⚪ Minimal · up to This change fixes incorrect scaled GEMM addressing for supported padded tensor views while retaining packed-layout behavior and adds targeted regression coverage. No merge-blocking risk remains. Sequence Diagram(s)sequenceDiagram
participant Test
participant ScaledMMEntry
participant GEMMCaller
participant CUTLASS
Test->>ScaledMMEntry: submit aligned or padded tensors
ScaledMMEntry->>ScaledMMEntry: validate A, B, and output strides
ScaledMMEntry->>GEMMCaller: dispatch scaled matrix multiplication
GEMMCaller->>CUTLASS: pass runtime leading strides
CUTLASS-->>Test: write strided output
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files. (2 skipped: 2 unsupported.)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Forward aligned runtime leading strides through the common C3x caller and the SM90 FP8 caller, including its swapped output layout. Add INT8/FP8 regression coverage for normal and swap_ab dispatches plus an explicit alignment rejection test. Assisted-by: OpenAI Codex <codex@openai.com> Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
1084637 to
20917ae
Compare
Summary
Fix SM90 tensorwise CUTLASS scaled GEMM addressing for aligned sliced or padded
tensor views by forwarding their actual leading strides instead of
reconstructing tightly packed strides from shape alone.
The operator entry point accepts row-major A/output and column-major B with
aligned padded leading dimensions. The common C3x caller discarded that
metadata, and the SM90 FP8 caller independently reconstructed packed strides.
Both cases could silently read or write incorrect addresses. This forwards the
runtime leading dimensions without adding tensor copies; packed inputs retain
the same stride values as before.
The SM90 FP8 caller needs separate handling when
swap_abis enabled: therow-major output is presented to the epilogue as a column-major transpose, so
its leading dimension belongs to the second stride component.
The SM100/SM120 and blockwise FP8 callers construct their own packed strides
and have the same defect. They are not fixed here because I have no SM100/SM120
hardware to validate on; they are tracked as remaining work on #55534.
The added A leading-stride check does not reject an existing supported packed
layout: B already requires
b.stride(1) % 16 == 0, which enforces alignedKfor packed inputs, and packed A has
a.stride(0) == K. It only rejects paddedA views whose leading dimension violates CUTLASS's
AlignmentAB = 16requirement. When
Kitself is unaligned, the Python wrapper already selectsthe Triton fallback.
Partially addresses #55534; remaining architecture-specific callers are
tracked on that issue.
Regression coverage
The former combined subset test is expanded across INT8 and FP8, normal and
swap_abdispatches (M=512andM=32), and independent padded-A, padded-B,and padded-output cases. A negative case verifies that a misaligned A leading
dimension is rejected at the operator boundary.
Validation
Environment: NVIDIA H800 PCIe (SM90), CUDA 13.0, PyTorch 2.13.0+cu130.
Model evaluations are not applicable: this is a low-level kernel addressing
fix, and the focused numerical comparison covers the affected output contract.
Duplicate-work check
I searched open PRs by issue number and by
CUTLASS scaled_mm leading strideand
cutlass_gemm_caller stride. No open PR addresses this bug. The keywordresults #33651 and #41834 concern different kernel/backend changes.
AI assistance disclosure
AI assistance was used to help inspect the CUTLASS/PyTorch stride contract,
prepare the patch and tests, and organize the validation evidence. I reviewed
the changed lines and validation results before submission.